Omni-Rig sensors: what can be done with a non-rigid vision platform?

We describe the principles of building a moving vision platform (a Rig) that once calibrated can thereon selfadjust to changes in its internal configuration and maintain an Euclidean representation of the 3D world using only projective measurements. Formally, we address the question of how to obtain an invariant 3D projective representation from a dynamic collection of cameras. We show that the maximal generality is reached when the rig consists of 5 cameras whose center of projection remain fixed relative to each other during the motion. In other words, the non-rigid component motion may consist of change of internal parameters and relative camera orientations. The configuration reduces to 3 views when the Rig is built using a single physical camera with half-mirrors (beam-splitters) for creating 3 distinct views. We also briefly discuss 2-view configurations using halfmirrors and the principle behind adapting the configuration to allow for zoom lenses in the system. The new research paradigm on non-rigid rigs (we term “Omni-Rig”) is applicable to the design of Vision-based sensors that after calibration can move in space while changing critical elements of their configuration — such as changing focus on the fly, zoom, relative camera orientation and inclination of focal plane to object’s surface orientation — without the need for recalibration, i.e., using only projective calculations throughout its motion.